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ABSTRACT 

This study examined practices of teachers in Wake County Public Schools (WCPSS) 
high school Algebra I classes. Regression analyses of standardized state testing 
results allowed for identification of the most effective and least effective Algebra I 
teachers and schools. The study used surveys, observations, and focus group 
interviews to compare and contrast most effective teachers with less effective teachers 
It found that the most effective algebra teachers averaged 68% of their time on new 
material daily; averaged seven transitions in ninety minutes; used sustaining 
feedback; had structured classes with a culture of mutual respect; told their students 
what to expect on tests and cautioned them about possible errors; emphasized 
problem solving and processes; and exhibited a sense of humor. These results can be 
used to motivate teacher and school imorovement efforts. 


INTRODUCTION 

The importanee of teachers is recognized by the National Council of Teachers of Mathematics 
(NCTM) in Principals and Standards for School Mathematics, which details requirements of 
effective teaching, including the requirement to continually seek improvement. “The 
improvement of mathematics education for all students requires effective mathematics teaching 
in all classrooms” (National Council of Teachers of Mathematics, Inc. [NCTM], 2000, p. 17). 
Several studies of student gains on standardized tests from one year to another have found the 
student’s assigned teacher to be the most influential factor (Rivkin, Hanushek, & Kain, 2001; 
Sanders & Horn, 1994; Sanders & Rivers, 1996; Wright, Horn, & Sanders, 1997). 

The importance of teachers in facilitating student success is also recognized by the Elementary 
and Secondary Education Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 
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2001 (NCLB). Under NCLB, every state must develop and implement a plan to ensure that all 
students will be taught by a “highly qualified teaeher” (U.S. Congress, 2001; sec.2101). 

Margaret Spellings, U. S. Seeretary of Education, stated in a letter dated October 21, 2005, 
“There is also evidence that states are improving the quality of their teaching forces.” She wrote 
that the U.S. Department of Education was committed to the goal of every child being taught by 
an HQT by the end of 2005-06, and then outlined the actions that would be taken to support 
states in reaching this goal (Spellings, 2005). Although the acknowledgement of the importance 
of high quality teachers is widespread and long-standing, defining the characteristics and 
behaviors that make a “high-quality” teacher remains an elusive goal. 

PURPOSE OF THE RESEARCH 

The main purpose of this research study was to improve the performance of WCPSS students in 
Algebra 1 by implementing a value-added instructional improvement analysis model. There 
were three specific objectives: 

• collect system-specific data to help teachers and district leadership understand current 
Algebra 1 instructional practices, 

• identify and share best practices in Algebra 1, and 

• contribute to a series of studies that identify the role of teachers, system staff, or departments 
in the school improvement process, and identify the practices of effective instruction. 

In 2005-06, 87.3% of WCPSS’ high school Algebra I students were proficient on the North 
Carolina End-of-Course (EOC) Algebra 1 exam. Students entering high school in 2006-07 and 
following must not only earn an Algebra 1 credit to graduate but must also pass the North 
Carolina State EOC exam to graduate (North Carolina Department of Public Instruction 
[NCDPl], 2005). Consequently, 13% of high school Algebra 1 students failed to meet the state 
standard. It is of increasing importance that teachers implement instructional practices that 
ensure all students success because of the importance of passing this course and test. 

Success in this study was defined using a multiple regression analysis of the state EOC 
standardized tests. This analysis generates for each student, teacher, and school a measure of 
whether their test scores show a level of performance that is either higher, lower, or about what 
is expected, compared to other WCPSS students, teachers, or schools. A previous study of 
biology teaching and learning demonstrated that it is possible to indentify teachers and schools 
that consistently produce more achievement in students regardless of a student’s initial skill 
level. (Haynie, 2006). This study brings value-added meaning to the Algebra 1 EOC test 
performance. The classroom practice of the most successful teachers can then be documented to 
give hope to those teachers struggling with low performers and to challenge the teachers of high 
performers to even higher academic goals. The school wide practices of successful schools serve 
as models for school improvement efforts. Unlike most current valued- added models, teacher 
performance evaluation was not a goal of this study (Braun, 2005; Olson, 2004a, 2004b; Olson, 
2005; Sanders, 1998; Tucker & Stronge, 2005). This study demonstrates an innovative use of 
value-added research for teacher and school improvement thus affecting improvement at the 
student level. 
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BACKGROUND 

STATE-DEFINED EFFECTIVENESS 

In North Carolina, EOC exams are administered in 10 high sehool subjeets, ineluding Algebra I. 
Each exam is a standardized multiple-choice test written with much input from teachers across 
the state. Teachers participate in test development in a variety of ways, from writing curriculum 
to writing and reviewing test items. Each student who takes an EOC test is assigned a scale 
score based on the number of items correct and the difficulty of items. The scale scores are then 
converted to one of four levels of performance. Eevels III and IV are associated with adequate 
or higher mastery of course content, and are considered proficient (NCDPI, 2005). 

Teachers receive rosters of students’ scale scores, level scores, and a 100-point scale score that is 
averaged as 25% of the final class grade, as required by State Board of Education policy. An 
average scale score for the class is also reported on each roster. The percentages of students 
passing each EOC in a school are reported publicly. Teachers judge their own success using 
these percentages. The scores can also be disaggregated into many subgroups (e.g. students with 
disabilities (SWD), free or reduced-price lunch (ERL) students, academically gifted (AG) 
students, etc.). 

If 90% of the EOC scores at a school are in Level III or IV, the school is eligible to be labeled a 
“School of Excellence” by the ABCs of Public Education, the state’s accountability program. 
However, the program has two standards of achievement: the absolute percentage of tests at or 
above grade level, and the attainment of “expected” growth. The basic assumption of the growth 
part of the model is that a student should be expected to do at least as well on each EOC test as 
he or she has done on prior End-of-Grade (EOG) and EOC tests compared to all other students 
who took the test in the standard-setting year. (The standard-setting year is typically the first 
year that a test becomes operational and students receive scores for the test.) Each student who 
is tested and has previous test results is assigned an “academic change” value. A positive 
“academic change” indicates academic progress, while a negative value indicates a loss of 
academic progress. The average of all students’ “academic change” values across all EOCs is 
calculated. If it is zero or higher, the school makes “expected growth”. If the school makes 
expected growth, all teachers receive a monetary bonus, regardless of the percentage of students 
testing at or above grade level. Teachers and schools with academically weaker students can still 
make “expected growth” regardless of the level performance of students and vice versa. 

Teachers with high- achieving students do not always produce “expected growth” in their 
students, but the “expected growth” measure is considered by most teachers to be a fairer 
measure of success than student proficiency alone (NCDPI, 2006). 

Algebra I is a North Carolina high school graduation course requirement. Starting with the 
entering freshmen class of 2006-07, students are also required to score proficient on the state 
Algebra 1 EOC exam. In 2004-05, 85.8% of high school Algebra I students in WCPSS scored 
proficient. The percentage of proficient students at the 17 WCPSS high schools ranged from 
68.8% to 93.1%, with two schools below the state average of 76.1%. The average scale scores 
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across schools ranged from 59.2 to 68.5. Two schools were below the state average of 63.1 (the 
state average ineluded middle sehool students). As Table 1 shows, 12 high sehools made high 
growth, four made expeeted growth, and one was below expeeted growth in Algebra 1 on the 
North Carolina ABCs aeeountability model. The sehools are labeled A to Q, with A having the 
highest pereentage of students scoring at Level 111 or IV and Q having the lowest pereentage. 
Also note that the sehool with the highest pereentage profieient (sehool A) is not the same as the 
sehool with the highest average seale seore (sehool E). 

Table 1 

2004-05 Algebra I Outcomes at 17 WCPSS High Schools 


School 

% at Level III or IV 

Average Scale Score 

NC Growth 

A 

93.1% 

67.8 

HIGH 

B 

92.9% 

67.0 

HIGH 

C 

91 .9% 

66.4 

HIGH 

D 

91 .8% 

68.3 

HIGH 

E 

91 .7% 

68.5 

HIGH 

F 

90.8% 

66.0 

HIGH 

G 

90.3% 

66.8 

HIGH 

H 

90.2% 

65.8 

HIGH 

1 

88.3% 

64.4 

HIGH 

J 

87.4% 

64.2 

HIGH 

K 

86.4% 

63.2 

HIGH 

WCPSS High Schools^ 

85.8% 

64.3 

HIGH 

L 

80.4% 

61.8 

EXPECTED 

M 

79.2% 

60.8 

EXPECTED 

N 

77.4% 

59.5 

EXPECTED 

0 

76.3% 

61.0 

HIGH 

State^ 

76.1%^ 

63.1 ^ 


P 

71 .6% 

59.0 

EXPECTED 

Q 

68.8% 

59.2 

BELOW 


^ Only high sehool students 

All NC Algebra I students (high school and middle sehool students) 

WCPSS-DEFINED EFFECTIVENESS 

The state provides test analysis software to every distriet in the state that ean be used to run 
sehool-level results and results for subgroups of students within the sehool. The state also posts 
these disaggregated test results on a Web \site. These analyses, however, are limited to average 
seale seores and pereentages of students tested who attain profieieney. Although these statisties 
provide useful information to teaehers and prineipals, WCPSS evaluators felt that other analyses 
of student aehievement data eould present a more useful picture of the sueeess of teaehers and 
students. While these measures serve as a valid way of reporting how teaehers and sehools are 
sueeeeding, WCPSS has developed more fine-grained methods to determine which schools are 
getting the most growth with students in comparison to other WCPSS schools. Identifying the 
schools and teachers that are produeing the highest performanee in students at varying levels of 
preparation is neeessary in order to share best praetiees within the distriet and motivate school 
improvement efforts. 
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Since the early 1990s, WCPSS has used a multiple regression analysis on state tests and has 
generated an effectiveness index for each school that ranks the schools within WCPSS by their 
effectiveness with the students who attend their school. The regression analysis creates a 
prediction model by using the current year’s test scores as the outcome and previous state test 
scores as the predictors. The analysis also takes into account each student’s special program 
status (level of service, e.g. self-contained), FRL status, and AG status, as well as the percentage 
of FRL students in the school. A residual score is calculated for each WCPSS student who took 
the test and had previous test scores predictors. The residual score for a student is the difference 
between the student’s actual score and the score that the regression analysis model predicted 
given the student’s previous test scores and program characteristics. These residuals give a 
measure of how students performed compared to other similar students (e.g. students with the 
same previous test scores and program identifiers) in WCPSS. Figure 1 gives a graphical visual 
of the regression analysis with three of the many possible lines. It shows that the predictions are 
dependent on the previous FOG and/or EOC scores and also the program characteristics of the 
student such as an AG student or a SWD student in a resource setting who is also a FRL student. 
The middle line could represent students with no identifiers (e.g. typical students). 

Figure 1 

Algebra I Regression Scatter Plot 
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Figure 2 shows one vertieal sliee of the regression model for the students whose predietion line is 
the middle line of Figure 1 and whose 8**'-grade FOG score was 170. The line at the center is the 
predicted score generated by the model for these students. A residual is the difference between 
the actual score and the predicted score. Scores above the line at the center have positive 
residuals and scores below have negative residuals. 


Figure 2 

Student Residuals for Average Students with No Program Identifiers 


Algebra scale score 75, residual 75-60 = 15 


Algebra scale score 60 




These scores have positive residuals. 


• ^ The predicted score from the regression model 

These scores have negative residuals. 


' grade EOG score of 170 


For each test given, the residuals are averaged across all students in the school, and a 
standardized z-score (“effectiveness index”) is generated for each school by subject. A z-score is 
the number of standard deviations that the school’s residual average is from the average (mean) 
of all the schools with at least 30 students who have test scores and previous scores. 

If the z-score is greater than 1 , then the students in that course at that school have scored 
significantly higher (among the top 16%) than other students in the district who have similar 
previous test scores and program characteristics. Similarly, if the effectiveness index is less than 
-1, then the students have scores much lower (among the bottom 16%) than other similar 
students. Values between -1 and +1 are within one standard deviation of the district average, 
and are considered “typical” or expected (68% of scores should be in this range). 
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Table 2 is an example of a WCPSS high sehool effeetiveness z-seore report. This sehool was 
among the top sehools in the distriet for effeetiveness in Algebra I, Algebra II, geometry, 
biology, and physieal seienee, yet the effeetiveness seores were among the bottom in 
oivios/eeonomies and U. S. history. The effeetiveness seores in English I, ehemistry, and physies 
were about the same as the average seores in the distriet. 

Table 2 

Effectiveness Indices for a WCPSS High School 
2005-06 


End of Course 

Z Score 

Algebra 1 

1.02 

Biology 

1.82 

Civics 

-1.00 

English 1 

0.32 

US History 

-1.44 

Algebra 2 

1.26 

Chemistry 

-0.25 

Physical Science 

1.37 

Physics 

-0.40 


WCPSS principals receive rosters of the student residuals by teacher, course, and section. In 
these rosters, student residuals above one standard deviation are coded in green, and student 
residuals below one standard deviation are coded in red. The standard deviation in scale score 
points of these residual scores is displayed at the bottom of roster, along with the average 
residual for the section. 
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Table 3 is a sample roster for a 2005-06 Algebra I elass of 17 students. The predietor seores are 
shown for eaeh student,. The predietors for the Algebra I EOC were the 8th-grade reading and 
mathematies seores. The roster then displays the EOC seale seore and the residual seore for eaeh 
student. These residuals are averaged and an average residual seore for the elass is provided. 

The average residual for this elass was 1.09. The prineipal and teaeher ean then determine how 
sueeessful students were on the EOC as eompared with other students with similar 
eharaeteristies. Notiee that Student 5 and Student 12 have the same seale seore on the Algebra I 
EOC exam, but Student 5 has a negative residual while Student 12 has a positive residual. 

Student 16 has a lower seale seore than Student 7 but a mueh higher residual. The residual 
shows a measure of performanee as related to previous performanee and other edueational 
indieators, and gives a sense of the relative growth for eaeh student. Student 1 1 has no residual, 
as previous test seores are missing. 


Table 3 

Sample High School Algebra I EOC Residual Roster 


Name 

8'*'-Grade EOG 
Reading Scale Score 

S^-Grade EOG 
Math Scale Score 

2006 

A1 

Scale score 

2006 

A1 

Residual 

Student 1 

260 

281 

63 

-7.52 

Student 2 

270 

285 

66 

-7.16 

Student 3 

269 

285 

70 

-6.39 

Student 4 

272 

273 

70 

-5.17 

Student 5 

272 

279 

72 

-1.16 

Student 6 

273 

285 

76 

-0.81 

Student 7 

277 

286 

77 

-0.23 

Student 8 

258 

275 

67 

0.41 

Student 9 

265 

267 

64 

2.04 

Student 10 

273 

283 

76 

3.00 

Student 1 1 

No seore 

No seore 

71 

No residual 

Student 12 

264 

275 

72 

4.71 

Student 13 

270 

281 

77 

5.50 

Student 14 

270 

280 

77 

6.11 

Student 15 

269 

286 

80 

6.20 

Student 16 

262 

277 

76 

8.35 

Student 17 

273 

282 

82 

9.61 


Note: 1 . Class Average = 1 .09 

2. Standard deviation = 5.94 

3. EOG is End of Grade exam and A1 is Algebra 

Interpretation Example: Student 14 had an 8‘**-grade reading scale score of 270 and an 8*-grade math scale score of 
280. Student 14’s 2006 Algebra I scale score was 77 and the student’s residual was 6.1 1. This student 
scored among the top 16% of students with the same Algebra I score, the same 8*-grade reading and 
math scores, and the same academic program indicators. 
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School Performance 

Table 4 shows the Algebra I effeetiveness indices of 16 WCPSS high schools in 2002-03, 2003- 
04, and 2004-05 (one of the previous 17 schools was new and did not have data for three prior 
years). Schools with an effectiveness index above 1 are coded as H for high and below a -1 as L 
for low. Schools with effectiveness indices between -1 and 1 are coded with an M. The schools 
were assigned random identification numbers, used throughout this study. The order of schools 
in no way corresponds to the order in Table 1. 

There are three noteworthy points: 

• School 6 had high Algebra I effectiveness for 3 years, 

• School 3 had low Algebra I effectiveness for 3 years, and 

• School 9 is falling in rank. 


Table 4 

School Level WCPSS Effectiveness Indices 


School 

2002-03 

2003-04 

2004-05 

1 

M- 

M- 

M- 

2 

M+ 

H 

H 

3 

L 

L 

L 

4 

M+ 

M+ 

M- 

5 

M- 

M- 

L 

6 

H 

H 

H 

7 

M- 

M+ 

M+ 

8 

L 

L 

M- 

9 

H 

M+ 

M- 

10 

M+ 

M- 

M- 

11 

M- 

M- 

M+ 

12 

H 

M+ 

M+ 

13 

M+ 

M+ 

M+ 

14 

M+ 

M+ 

M- 

15 

M- 

M- 

M- 

16 

M+ 

M+ 

M+_ 


Note: 


1. H = effectiveness index >1 

2. L = effectiveness index < 1 

3. M+ = effectiveness index between 0 and 1 

4. M- = effectiveness index between -1 and 0 


Teacher Performance 

The student residual scores and the effectiveness indices give the district a comparison basis for 
schools and students. Until recently, residuals had not been averaged or standardized at the 
teacher level beyond the classroom roster. Prior to 2005-06, teachers were encouraged to study 
their rosters for trends in student performance, and some principals compared teachers within 
their school, but no district wide comparisons were made. The study of biology teaching by 
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Evaluation and Research (E&R Department) staff in 2004-05 was a first attempt at identifying 
the success of teachers, as indicated by average residuals, and then to identify the specific 
aspects of the practice of highly effective and relatively less effective WCPSS teachers in order 
to isolate teachers’ classroom practices that may be associated with high student achievement 
(Haynie, 2006). This study of Algebra I outcomes and practices is the second in a planned series 
of studies that will target overall systematic improvement as well as individual subject 
improvement using the residual metric and recognizing that teaching is an essential unit of 
analysis. 


METHOD 

The biology study piloted a collaborative study between the WCPSS Curriculum/Instruction 
Department (C&I) and the E&R Department (E&R) with the goal of identifying best teaching 
practices. Since both the effectiveness indices and teacher residual averages use residual values 
that are calculated using student test results, that are known to contain error, combining 
residuals over three consecutive years of data both reduced the test error and removed 
inexperienced teachers and teachers who taught biology infrequently. This piloted model was 
deemed successful. The teachers of the biology study had consistent residual results over the 
span of four years. Eor the biology study, the specialist from C&I took the lead in interpreting 
classroom observation while the specialist from E&R took the lead in data collection and 
analysis. This basic methodology was used again in this Algebra I study. 

CONTEXT AND PARTICIPANTS 

This research study took place in WCPSS, a large urban/suburban school district in North 
Carolina. The WCPSS student population is growing rapidly, with an enrollment of 
approximately 128,000 in 2005-06. There were 17 high schools in the district, of which 15 were 
using a block schedule with two semesters per year. One school was on an alternating-day block 
schedule (a course is 18 weeks in length but taught throughout a full school year). One other 
school was on a seven-period full-year schedule with 50-minute classes. 

In 2005-06, there were 157 high school Algebra I teachers. Sixteen of these teachers were first- 
year teachers and eight were lateral entry teachers. Eorty-one teachers (26% of the 157 teachers) 
had taught Algebra I in 2002-03, 2003-04, and 2004-05 and were teaching in 2005-06, which 
made them eligible for this study. Eor these 41 teachers at 1 1 schools, the average student 
residual across all years and classes was calculated. The teachers were ranked on teacher 
effectiveness from highest to lowest using these averages. The teachers with the nine highest 
residual averages were labeled T1-T9 and the teachers with the nine lowest averages were 
labeled B1-B9. The practice of these 18 teachers became the ultimate focus of this study. 

Table 4 gives the residual average (in scale score points), the mean of the scale scores, the 
overall percentage of students who scored proficient, and the percentage of white students and 
Black/ African American students who scored proficient for the 1 8 teachers studied. The school 
code used is the same as in Table 3. 
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Table 5 demonstrates several points: 

• the top nine teaehers were in seven different sehools, 

• the bottom nine teaehers were in five sehools, 

• sehool 14 had both a top and a bottom teaeher, 

• residual averages ranged from -6.72 to 4.77, 

• the aehievement gap between White and Blaek/Afriean Ameriean students was mueh higher 
for most bottom teaehers than top teaehers, 

• top teaeher 1 (Tl) and top teaeher 2 (T2) taught at the same sehool, and 

• the bottom four teaehers (Bl- B4) taught at the same sehool. 

Also note that top teaeher 3 and bottom teaeher 3 have identieal pereentages of profieient 
students, but very different residual averages. This shows the value added by the residual 
analysis, whieh aeeounts for how students seore in eomparison to students with similar previous 
test performanee and edueational indieators. The higher residual average of top teaeher 3 
indieates a higher level of student performanee relative to other students. Also, it is of interest 
that top teaeher 2 (T2) mentored top teaehers 1 and 3 (Tl and T3). 

Table 5 

Residual Averages for Top and Bottom Study Teachers 


TEACHER 

SCHOOL 

RES AVG 

AVG SS 

% PROF 

% WH PROF 

% BK PROF 

ACH Gap 

T1 

7 

4.77 

71.6 

94 

96 

89 

7 

T2 

7 

4.27 

62.4 

92 

100 

100 

0 

T3 

9 

3.83 

62.4 

89 

92 

90 

2 

T4 

2 

3.57 

70.4 

97 

99 

92 

7 

T5 

1 

3.37 

68.5 

96 

94 

96 

-2 

T6 

4 

3.26 

71.8 

97 

100 

95 

5 

T7 

14 

3.17 

74.1 

100 

100 

100 

0 

T8 

6 

3.01 

68.6 

100 

100 

100 

0 

T9 

6 

2.95 

72.2 

100 

100 

100 

0 

B9 

8 

-1.74 

61.2 

78 

88 

69 

19 

B8 

14 

-2.21 

63.2 

91 

96 

85 

11 

B7 

15 

-2.22 

63.4 

87 

96 

74 

22 

B6 

8 

-2.45 

61.7 

77 

90 

66 

24 

B5 

5 

-3.72 

63.0 

85 

93 

75 

18 

B4 

3 

-4.80 

58.9 

78 

86 

75 

11 

B3 

3 

-5.31 

55.5 

89 

92 

90 

2 

B2 

3 

-5.91 

57.5 

69 

95 

57 

38 

B1 

3 

-6.72 

56.3 

55 

87 

37 

50 


Note: The overall percent proficient includes American Indian, Asian, Hispanic, and Multiracial 

students. 

Interpretation Example: Teacher T3 teaches at School 9. T3 had a student residual average of 3.83 for all 
the students of 2002-2005. The average scale score for these students was 62.4.; 89% of these 
students scored at Level III or Level IV (proficiency); 92% of the White students and 90% of 
the Black/African American students scored proficient. 
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INSTRUMENTS, DATA COLLECTION, AND ANALYSIS PROCEDURES 

In addition to these student achievement scores, three other types of data were collected for this 
study. First, a survey was prepared and distributed to the 41 study teachers. The teachers 
answered 37 written survey questions concerning preparation, planning, use of time, schedules, 
use of data, and student interaction. 

Second, one of the two principal investigators for this study observed 15 of the 18 teachers 
identified as most effective (“top teachers”) or least effective (“bottom teachers”). One of the 
top teachers was very ill throughout much of the observation year, one bottom teacher never 
became available, and only three of the four bottom teachers at the same school were observed. 
The observation instrument was adapted from the one previously used in the biology study 
(Haynie, 2006). Both of the observers were trained in supervisory observations. 

Third, school and teacher focus-group interviews were conducted. There were four school focus 
groups: Two at the schools with the highest 2004-05 school effectiveness indices, and two at the 
schools with the lowest 2004-05 school effectiveness indices. There were also teacher focus- 
group interviews of the top teachers and the bottom teachers. The same questions were explored 
at all six focus-group interviews. 
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RESULTS 


OVERALL 

Student Academic Achievement 

Prior performance of students on 8th-grade reading and mathematics EOG state tests was 
averaged for each of the top and bottom teachers. Table 6 shows the averages and the standard 
deviations. The overall average 8th-grade mathematics scale score ranged from 264.9 to 278.6 
for top teachers and 266.7 to 274.3 for bottom teachers, with the standard deviations ranging 
from 6.5 to 9.5 for top teachers and from 7.4 to 10.1 for bottom teachers. The overall average 
was 273.8 for top teachers and 271.5 for bottom teachers, with an overall standard deviation for 
top teachers of 7.5 and 8.4 for bottom teachers. 

Table 6 

8th-grade Student EOG Reading and Mathematics Averages 

2002-2005 


TEACHER 

SCHOOL 

n 

Residual 

Average 

8'"-grade 
reading 
scale score 
average 

8'"-grade 

reading 

standard 

deviation 

8'"-grade 
math scale 
score 
average 

8'"-grade 

math 

standard 

deviation 

T1 

7 

504 

4.77 

266.7 

6.3 

274.3 

7.1 

T2 

7 

105 

4.27 

261.8 

6.7 

266.9 

6.7 

T3 

9 

370 

3.83 

260.7 

6.9 

264.9 

7.3 

T4 

2 

350 

3.57 

266.6 

6.8 

275.3 

7.6 

T5 

1 

318 

3.37 

265.5 

7.2 

273.0 

7.8 

T6 

4 

335 

3.26 

267.9 

6.9 

276.9 

9.5 

T7 

14 

205 

3.17 

270.0 

4.2 

278.6 

6.5 

T8 

6 

157 

3.01 

264.8 

7.6 

272.9 

7.5 

T9 

6 

346 

2.95 

268.7 

6.1 

278.4 

6.6 

B9 

8 

124 

-1.74 

266.2 

6.6 

270.2 

10.1 

B8 

14 

360 

-2.21 

264.6 

6.6 

271.3 

7.7 

B7 

15 

242 

-2.22 

267.1 

6.9 

274.3 

7.4 

B6 

8 

88 

-2.45 

265.3 

5.5 

271.4 

8.0 

B5 

5 

290 

-3.72 

265.5 

6.6 

273.8 

8.5 

B4 

3 

129 

-4.80 

264.3 

5.3 

270.9 

8.1 

B3 

3 

124 

-5.31 

261.5 

8.8 

266.7 

9.9 

B2 

3 

150 

-5.91 

264.8 

7.9 

270.8 

8.7 

B1 

3 

120 

-6.72 

264.3 

7.6 

268.9 

9.1 


Note: m’s include only students with residual scores (e.g. 8*-grade EOG scores and an Algebra I EOC score) 
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Several factors might account, in part, for the results reported in Table 6; 

• general academic preparation of students, 

• experience level of teachers, 

• the teaching assignments of teachers. 

We examined each of these. 

Academic Preparation of Students 

The bottom teachers of this study had not only an overall lower 8th-grade mathematics scale 
score average but also a higher overall standard deviation. Note though that top teachers 2 and 3 
had students with low 8th-grade EOG performances and top teacher 6 had a large standard 
deviation. Also, there were top teachers who exhibited success with both higher- and lower- 
achieving groups in terms of their scores in 8th-grade. The residual average controls for the 8th- 
grade scores and gives a comparison of teachers based on their students’ performance regardless 
of the academic preparation of students. 

Teacher Experience and Course Load 

Survey results showed that there was no striking difference between the experience levels of the 
top and bottom teachers. The years of teaching experience for top teachers ranged from 4 to 33 
years, with 4 to 17 of those years in Algebra I. The bottom teachers had years of teaching 
experience that ranged from 4 to 20 years, with 4 to 20 in Algebra I. 

The number of Algebra I contacts (students) that top teachers taught in three years ranged from 
105 to 504, with 88 to 360 for bottom teachers. The number of mathematics teaching 
preparations per year ranged from two to four for both top and bottom teachers. Finally, the 
percentage of Algebra I course sections that were taught by each teacher was calculated. These 
courses were Algebra I, Algebra I-Part 1, Algebra I-Part 2, Algebra I with Technology-Part 1, 
Algebra I with Technology-Part 2, and Algebra I Plus. Overall, top teachers were teaching more 
Algebra I, with seven teachers above 50% of their courses and five of the seven above 75%, 
compared to three above 50% and only one above 75% for bottom teachers. 
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Table 7 shows the details of the experienee levels. Note that top teaeher 2 and bottom teaeher 2 
illustrate similar experienee despite very different residual averages. 


Table 7 

Professional Factors of Study Teachers 


TEACHER 

SCHOOL 

# of years 
teaching 

# of years 
teaching 
Algebra 1 

# of Algebra 1 
students in 
residual average 

# of preps per 
year for 2002-03, 
2003-04, 2004-05 

% of course sections taught 
from 2002 to 2005 that use 
Algebra 1 curriculum 

T1 

7 

11 

9 

504 

2 , 2,4 

94 % 

T2 

7 

33 

15 

105 

3 , 3,4 

28 % 

T3 

9 

4 

4 

370 

not available 

56 % 

T4 

2 

28 

17 

350 

3 , 3,2 

89 % 

T5 

1 

8 

7 

318 

2 , 3,3 

76 % 

T6 

4 

19 

16 

335 

2 , 3,3 

88 % 

T7 

14 

16 

16 

205 

3 , 3,4 

56 % 

T8 

6 

16 

14 

157 

2 , 3,2 

41 % 

T9 

6 

5 

5 

346 

2 , 3,4 

82 % 

B9 

8 

17 

12 

124 

4 , 3,4 

32 % 

B8 

14 

8 

8 

360 

2 , 2,2 

100 % 

B7 

15 

4 

4 

242 

2 , 4,2 

59 % 

B6 

8 

4 

4 

88 

2 , 2,2 

50 % 

B5 

5 

7 

7 

290 

2 , 3,2 

67 % 

B4 

3 

11 

11 

129 

3 , 3,2 

44 % 

B3 

3 

10 

10 

124 

3 , 3,4 

38 % 

B2 

3 

20 

20 

150 

2 , 3,3 

33 % 

B1 

3 

5 

5 

120 

2 , 3,3 

38 % 


There were also similarities between top and bottom teaehers made during the observations. In 
most observations, students were on task with teaeher-led, skill-driven, aetivities, based on the 
North Carolina Standard Course of Study (NCSCS). 


Marzano’s Effective Practices 


During the 2004-05 sehool year, all high sehool teaehers and administrators took part in sehool- 
level book studies. The book of ehoiee for all sehool staff was Marzano, Piekering & Polloek’s, 
Classroom Instruction that Works (2001). The book study groups met at various times 
throughout the year and foeused on the effeetive instruetional strategies offered in the text. 
Beeause the teaehers were familiar with instruetional strategies deemed effeetive by Marzano et 
ah, it followed that this study would analyze implementation of these instruetional strategies. 
Items on the teaeher survey addressed the teaehers’ level of implementation {daily, often, 
sometimes, or never) of eaeh of the strategies identified and deseribed in Classroom Instruction 
that Works. Observations of teaehers either eonfirmed or eontested the implementation level of 
daily use as self-reported on the teaeher survey (see Table 8). 

Marzano et al. (2001) used meta-analysis to identify instruetional strategies and ealeulate for 
eaeh an effeet size (a measure of the differenee in aeademie performanee of groups that used the 
partieular strategy eompared to those that did not). The meta-analysis eombined many studies of 
the use of partieular instruetional strategies. 


15 































































































































Effective High School Algebra I Teaching 


E & R Report No. 06. 10 


Nine strategies were found to be most effeetive; 

1. Identifying similarities and differenees. 

2. Summarizing and note taking. 

3. Reinforeing effort and providing recognition. 

4. Homework and practice. 

5. Nonlinguistic representation. 

6. Cooperative learning. 

7. Setting objectives and providing feedback. 

8. Generating and testing hypotheses. 

9. Questions, cues and advanced organizers (Marzano et al, 2001, p. 7). 

The above strategies are presented in order of effectiveness on student achievement. Identifying 
similarities and differences was found to be the most effective strategy. Marzano et al. (2001) 
suggest that there are four ways for teachers to guide students in identifying similarities and 
differences: 

• comparing, 

• classifying, 

• creating metaphors, and 

• creating analogies (Marzano et ah, 2001, p. 13-28). 

Three top and two bottom teachers indicated on the surveys that they used similarities and 
differences daily. Of these five teachers, one top and one bottom teacher were observed using 
the strategy. In addition, two other top and one bottom teacher were observed using similarities 
and differences, for a total of 5 out of 15 teachers (one-third), with no observed difference 
between top and bottom teacher use. 

Summarizing and note taking are cited as the second most effective strategy. These activities 
help students identify and understand the most important ideas in an academic discipline. Both 
of these activities need to be taught in a way that places purpose in the activity. Students need 
frameworks for both summarizing and note taking (Marzano et ah, 2001, pp. 29-48). The 
teacher survey separated these into two items. Two top and five bottom teachers reported that 
they summarize daily. One member of each group was observed summarizing. Five top and five 
bottom teachers reported using note taking daily. Of these teachers, two top and two bottom 
teachers were observed using note taking. Both top teachers were providing meaning for the 
note taking, but only one of bottom teachers was. There was one additional bottom teacher 
observed summarizing, for a total of 4 out of 15. There was also another bottom teacher 
observed using meaningful note taking, for 5 out of 15. The percentage of use was about one- 
third for both groups, but bottom teachers reported more use of both strategies (four of seven) 
than was observed. 

The third strategy is reinforcing effort and providing recognition. Recognition needs to be tied to 
specific goals so that students learn the value and reward of hard work. Praise can be ineffective 
if delivered randomly and globally. Effective praise is tied to the attainment of a specified 
performance of a task difficult for a student (Marzano et ah, 2001, pp. 49-59). This strategy was 
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also divided into two items on the teaeher survey. Four top and seven bottom teaehers elaimed 
to reinforee effort daily. Of these, three top and two bottom teaehers were observed doing so as 
defined by Marzano. Of the teaehers elaiming to provide reeognition daily, two of three top 
teaehers and three of five bottom teaehers were observed doing so. In addition to the teaehers 
reporting daily use, two more top teaehers were observed reinforeing effort and four more 
providing reeognition, for a total of five out of eight and six out of eight teaehers; eompared to a 
total of two out of seven and five out of seven bottom teaehers. Top and bottom teaehers 
provided reeognition, but top teaehers were observed reinforeing effort more. Although all 
bottom teaehers observed reported reinforeing effort, only two of the seven did so as defined by 
Marzano where the reinforeement is elearly tied to a speeifie, diffieult goal. 

The fourth strategy is homework and praetiee. This strategy is widely used, but not always in the 
most effeetive way. In order for the homework and praetiee to be meaningful, the purpose must 
be elearly eommunieated to students. The results need to be eonneeted to elassroom objeetives. 

It ean be used to praetiee a skill or introduee a new eoneept. Either way, the student should 
understand the purpose and reeeive feedbaek on the homework (Marzano et ah, 2001, pp. 60-71). 
This strategy was observed being used by all top teaehers, but one top teaeher did not provide 
meaningful feedbaek on the assigned homework. Of the bottom teaehers, four of nine were 
observed using it eorreetly. Three other teaehers said they did, but were not observed using any 
homework. All top teaehers were observed using guided praetiee and all but one used 
meaningful homework. Three of seven bottom teaehers reported using this strategy, but were not 
using it as defined by Marzano. 

Nonlinguistie representations is the fifth strategy (Marzano et ah, 2001, pp. 72-83). Sinee the 
Algebra I eurrieulum emphasizes three modes of representation (equation, table, and graph) for 
most topies, the observers expeeted to see this strategy used. Yet only one top teaeher elaimed to 
use this strategy daily. Four of eight top teaehers and two of seven bottom teaehers were 
observed using nonlinguistie representations, for a total of 6 out of 15 (40%). 

One bottom teaeher elaimed to use eooperative learning, but it was not found in observation 
(Marzano et ah, 2001, pp. 84-91). Two top teaehers used eooperative learning during the 
observations, for a total of 2 out of 15. 

The seventh strategy is setting objeetives and providing feedbaek (Marzano et ah, 2001, pp. 92- 
102). In a elass sueh as Algebra I where there is a eurrieulum that will be tested by a state EOC 
exam, it is easy to assume that everyone knows what the objeetives are. Only four top and four 
bottom teaehers marked on their surveys that they provided daily objeetives. Of these, only one 
top teaeher was observed doing so. This was the only 1 of 15 teaehers observed providing 
objeetives. Marzano states that having goals foeuses instruetion. The goals need to be flexible 
so that students ean personalize them. They also need to be broad enough to eneompass 
enriehing topies. 

The feedbaek that students reeeive should be refereneed to the stated objeetives. It needs to be 
timely, speeifie, and eorreetive in nature. Seven of eight top teaehers and six of seven bottom 
teaehers said they gave daily feedbaek. Of these, five top and two bottom teaehers were 
observed doing so. There was one additional top and one additional bottom teaeher observed 
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giving meaningful feedback. Top teachers were observed using meaningful feedback more than 
bottom teachers (six of eight, compared to three of seven). 

Generating and testing hypotheses are often considered part of the scientific process and so 
should be part of all mathematics classes (Marzano et ah, 2001, pp. 103-110). Only two top and 
one bottom teacher were observed using this method. The one bottom teacher who claimed to do 
so daily was not observed doing so. Six of eight top and six of seven bottom teachers were 
observed using cues, questions, and advanced organizers (Marzano et ah, 2001, pp. 103-120). 

Yet questioning was at a much higher level in the classes of top teachers. They probed students’ 
conceptual understanding and the whys behind answers; bottom teachers questioned the steps 
needed in skills. Top teachers also involved more students during their questioning. Top 
teachers provided advanced organizers in their warm-up exercises. 

Both top and bottom teachers reported more use of Marzano strategies than was observed. Most 
teachers were providing recognition, as well as asking questions, providing cues, and using 
advanced organizers. Most top teachers were also reinforcing effort, using guided practice, and 
providing meaning to homework. Less than one-third of the teachers were using similarities and 
differences, summarizing, note taking, cooperative learning, providing objectives, or generating 
and testing hypotheses. 
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Table 8 summarizes the observations of Marzano strategies. 

Table 8 

Observation Results of Marzano Strategies 


Marzano Strategy 

Observed out 
of 8 top 
teachers 

Observed out 
of 7 bottom 
teachers 

Observed of top 
teachers who 
claimed daily use 

Observed of bottom 
teachers who 
claimed daiiy use 

1. Identifying Simiiarities 
and Differences 

3 

2 

1 out of 3 

1 out of 2 

2. Summarizing 

2 

2 

1 out of 2 

1 out of 5 

2. Note Taking 

2 purpose 

2 purpose 
1 no purpose 

2 out of 5 

2 out of 6 

3. Reinforcing Effort 

5 

2 

3 out of 4 

2 out of 7 

3. Providing Recognition 

6 

5 

2 out of 3 

3 out of 5 

4. Homework 

7 purpose 
1 no purpose 

4 

8 out of 8 

3 out of 6 

4. Guided Practice 

8 

4 

6 out of 6 

4 out of 7 

5. Noniinguistic 
Representation 

4 

2 

1 out of 1 

none claimed 
daily usage 

6. Cooperative Learning 

2 

0 

none claimed daily 
usage 

0 out of 1 

7. Providing Objectives 

1 

0 

1 out of 3 

0 out of 4 

7. Providing Feedback 

6 

3 

5 out of 7 

2 out of 6 

8. Generating and Testing 
Hypotheses 

2 

1 

none claimed daily 
usage 

0 out of 1 

9. Questions, Cues, and 
Advanced Organizers 

6 

6 

2 out of 2 

5 out of 5 


Interpretation Example: Of the 15 teachers observed, 8 out of 8 top teachers and 4 out of 7 bottom teachers 

were observed using homework. One of the top teachers did not provide a purposeful context for the 
homework. Of the teachers who had marked their surveys that they used the homework strategy 
daily, 8 out of 8 top teachers were observed doing so, but only 3 out of 6 bottom teachers were 
observed doing so. 

TOP TEACHER BEHAVIORS 

Despite the similarities between top and bottom teaehers, some noteworthy differenees were 
found. One of the key differenees observed during classroom visits was the use of class time; 
top teachers spent much more time on average on new material. Table 9 summarizes the 
observations about time. 

Table 9 

Classroom Use of Time 


Content 

TOP 

BOT 

Old 

Material 

22% 

36% 

New 

Material 

68% 

36% 

Wrap-up 

10% 

28% 
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One of the principal investigators of this study is the senior administrator for secondary 
mathematics. Table 10 lists classroom behaviors that she observed in the classrooms of six top 
teachers, contrasted with the observations of five bottom teachers. Some of these behaviors are 
Marzano strategies that were noted in Table 8, and others are additional noteworthy findings. 

This listing summarizes behaviors that were present in most top teacher classes and missing from 
most bottom teacher classrooms. All six top teachers had created a classroom culture in which 
students were free to ask questions, contribute, or offer explanations within a structured 
classroom. Mutual respect was exhibited between teacher and student. Two of the five bottom 
teachers had a structured management style with mutual respect evident, but none of the bottom 
five had an open culture in which students freely contributed. All six top teachers used sustained 
feedback (detailed and complete information that students can understand and use to modify 
their thinking), but none of the bottom teachers gave this type of feedback. Top teachers took 
time to tell students what types of questions to expect on tests and assignments, while none of the 
bottom teachers were observed doing so. Top teachers cautioned students about possible errors 
commonly made; only one bottom teacher was observed doing so. Top teachers used a variety 
of activities with frequent transitions (averaging 7 in 90 minutes) and exhibited a sense of humor 
that added a feeling of fun to the classroom. Two of the five bottom teachers used variety and 
humor. All six top teachers took time to emphasize the importance of the effort that was needed 
to complete assignments and to study. Three of the five bottom teachers did this as well. 

Five of the six top teachers were observed questioning different students at each step of a 
problem and using a class warm-up exercise (a task given at the beginning of class) that spiraled 
to include prerequisite skills necessary for success in the current day’s new material. Five of six 
top teachers were also observed giving purpose to homework (connecting it carefully to the 
current lesson and using the homework in the lesson explanations). None of the five bottom 
teachers was observed using these behaviors. 

Four of the six top teachers were observed using explanations that were more concept-driven 
than skill-driven. They emphasized the why behind processes and indicated strategies for 
approaching the problem instead of memorizing step-by-step algorithms. None of the five 
bottom teachers was observed explaining concepts and emphasizing processes. 
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Table 10 

Top Teacher Behaviors 


BEHAVIOR 

TOP 

BOTTOM 

USED BY 6 OUT OF 6 TEACHERS 



Classroom culture In which students were free to ask 
questions, contribute, or offer explanations 

6 out of 6 

0 out of 5 

Sustained feedback 

6 out of 6 

0 out of 5 

Told students what to expect on tests and assignments 

6 out of 6 

0 out of 5 

Cautioned students about possible errors 

6 out of 6 

1 out of 5 

Variety of activltles-frequent transitions 

6 out of 6 

2 out of 5 

Structured classroom management style with mutual respect 

6 out of 6 

2 out of 5 

Teachers exhibited a sense of humor 

6 out of 6 

2 out of 5 

Emphasized effort In completing assignments and studying 

6 out of 6 

3 out of 5 

USED BY 5 OUT OF 6 TEACHERS 



Questioned different students at each step of problem 

5 out of 6 

0 out of 5 

Class warm-up spiraled to Include prerequisite skills for the 
day's lesson 

5 out of 6 

0 out of 5 

Gave purpose to homework 

5 out of 6 

0 out of 5 

USED BY 4 OUT OF 6 TEACHERS 



Explanations were concept-driven 

4 out of 6 

0 out of 5 

Indicated strategies for approaching problems rather than 
memorizing processes 

4 out of 6 

0 out of 5 


There were three other differenees of note in the survey responses of top and bottom teaehers. 
First, in response to the instructional mode of delivery, top teachers reported more use of class 
time in small groups and with technology than did the bottom teachers. Three of the four bottom 
teachers reported 0% of class time in small group, and the fourth teacher reported 5%. Of the 
other bottom teacher responses, only three were above 10%, whereas only one of nine top 
teachers responded 0% and six of nine responses were greater than 10%. All top teachers 
reported using technology, with eight of nine responses at 10% or more, while one bottom 
teacher reported no use of technology and only four reported at least 10%. 

Second, teachers were asked to rank algebra topics in order of importance in their algebra 
classes. Both top and bottom teachers chose solving equations as most important, but top 
teachers ranked linear regression much higher than bottom teachers. Out of 14 topics, eight of 
nine top teachers ranked it eight or higher, while seven of nine bottom teachers ranked it 10 or 
lower, with four bottom teachers giving it the lowest possible ranking. 

Third, most top teachers (eight of nine) reported planning with other teachers, while six of nine 
bottom teachers planned alone. In response to how common planning time was used, all the top 
teachers chose pacing as either first or second in importance. Six of nine top teachers set their 
own pace, while five of nine bottom teachers used a district-prepared pacing guide. 

Both top and bottom teachers stated that their students had weak prerequisite skills. Yet top 
teachers remediated within material with a spiraled curriculum while bottom teachers remediated 
before beginning new skills. Eight of nine top teachers stated that their Black/ African American 
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students performed at or above expeetations, while five bottom teaehers stated that they 
performed below expeetations. 

An interesting result was that seven of nine top teaehers disagreed with the statement “My 
students seek understanding of mathematies eoneepts,” while six of nine bottom teaehers agreed 
with the statement. Informal follow-up with the top teaehers found that they had a high level of 
expeetation for their students and never felt that they were reaehing high enough. These teaehers 
always pushed their students past their students’ eomfort level with eoneepts. 

There were also differenees in the foeus-group interview responses of top teaehers and bottom 
teaehers, as well as top sehools versus bottom sehools. The top teaehers responded with 
systemie eoneerns, sought to learn of others’ programs, and shared their ideas for improvement. 
Bottom teaehers responded with personal and management eoneerns (Fuller, 1969). For 
example, bottom teaehers spent mueh time diseussing the laek of motivation of their students, 
their unwillingness to do homework, student’s laek of prerequisite skills, and their own dislike of 
an EOC exam that asked questions that the students had not studied. Top teaehers diseussed 
their strategies for remediating prerequisites and how their programs taught the NCSCS. 

TOP SCHOOL-LEVEL BEHAVIORS 

Top sehool behaviors were identified using the answers from sehool foeus-group interview 
questions. There were four sehool-level foeus group interviews (two at the sehools with the 
highest 2004-05 sehool effeetiveness indiees, and two at the sehools with the lowest 2004-05 
sehool effeetiveness indiees). All teaehers who taught Algebra I were ineluded regardless of 
their number of years of experienee. 

Experienced Course Leader 

The two sehools with the highest 2004-05 Algebra I effeetiveness seores had a strong 
experieneed eourse leader. These teaehers took lead in seeing that their departments had a 
sehool wide plan eonsistent aeross elasses and aligned to the NCSCS. The sehool Algebra I 
leaders eneouraged the Algebra I teaehers to believe that they eould understand the state 
eurrieulum and find a way for their students to be sueeessful. 

The laek of a strong leader was evident at the bottom sehools. The teaehers at the two bottom 
sehools spent mueh of their time eomplaining about the state eurrieulum and giving exeuses as to 
why their students eould not sueeeed. The teaehers at one of the bottom sehools rejeeted the 
EOC test and viewed it as an inappropriate measure for their Algebra I eourses. One teaeher (the 
department ehair) said “The Algebra I EOC should be a skills test.” A new teaeher asked, 
“Shouldn’t it test what they know?” Another teaeher asked, “How ean they answer questions 
worded in ways that they have never seen? They don’t ask the questions the way they are 
taught.” There was no teaeher leader at this sehool who explained and supplied information to 
help both interpret the state eurrieulum and to say that it is the teaeher’s job to study and make 
the state eurrieulum aeeessible to students. 
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A School Wide Plan 

Both top schools prepared eomplete eourse materials during the summer prior to the first year of 
implementation of a new revised version of the Algebra I NCSCS. Both sehools write their own 
paeing guides using the district pacing guide as a model. At one top school, the sehool wide plan 
is for all Algebra I teaehers to use the same daily homework assignments. At the other top 
sehool, a eommereial eurrieulum with a book and supplemental materials is used, but the paeing 
guide is a school-prepared one and many additional lessons are written in order to fully address 
the NCSCS, espeeially the teehnology requirements. Both sehools have spiraled eurrieulum that 
remediates weak prerequisite skills within the new material. Both sehools emphasize problem 
solving and proeesses. 

Both top sehools used materials thoughtfully. The sehool that had eornmon homework 
assignments only used a textbook as a referenee. The teaehers said that the NCSCS was the 
curriculum, not the textbook. The teaehers at this sehool were also emphatic about using 
teacher-made tests, not tests supplied with the book. The teachers at the other sehool felt that 
many of the prepared tests supplied with their book were appropriate to use, but they have 
seleeted the text earefully. Both sehools withheld ealeulator use at the beginning of their 
eourses; they wanted to build eonfidenee in basie skills first, and introduee the ealeulator as a 
tool that supplements, but does not replaee, human thought. 

Both top sehools had support struetures for teaehers, with speeial eonsideration of new teaehers; 
they handed teaehers the prepared materials for the entire eourse. They also had experieneed 
teachers meet with new teaehers to listen to their eoneerns, explain the materials, and help with 
test writing. 

All the teachers of these top sehools expressed a pride in their department and felt part of a 
sueeessful team. One of the bottom schools had no team planning, while the other sehooTs 
teaehers spent mueh of their planning time expressing negative opinions toward any suggestions 
for ways to improve (there were many “yes, but” comments made). Teaehers at this sehool said 
that they planned together, but there was no eonerete evidenee of a sehool wide plan. When 
asked by their prineipal to show weekly lesson plans, they seemed overwhelmed at how to fulfill 
this request. 


CONCLUSIONS 

The main goals of this researeh study were to eolleet system-speeifie data to help teaehers and 
distriet leadership understand eurrent Algebra I praetiees, identify and share best instruetional 
praetiees in Algebra I, eontinue to build a series of studies that identify the role of teaehers, and 
other system staff/departments in the sehool improvement proeess, and identify the praetiees of 
effeetive improvement. 

It ean be eoneluded that in 2005-06, eight WCPSS high schools (47%) had more than 90% of 
their students seore profieient (at Level III or IV) in Algebra I. Fifteen high sehools (88%) 
seored above the state profieiency of 76.1%, and 12 high sehools (71%) made the state high 
growth measure. These pereentages all seemed high. Yet the sehools had from 7% to 3 1% of 
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their Algebra I students seoring below profieieney. The highest scale-seore average was 68.5, 
well below the highest scale score possible of 96. Few WCPSS high school students scored in 
the top half of Level IV. There was much room for improvement at all schools. 

The WCPSS effectiveness indices for schools and the average residual scores for teacher identify 
schools that and teachers who can be used to document best instructional practices in Algebra I. 
There are highly effective teachers in schools that are ineffective overall. There are schools that 
are very effective overall and have a clear school wide plan for success. There are teachers 
motivating low-performing students to success in Algebra I. Yet in this study, there was one 
school that had four teachers with low effectiveness and that overall was less effective than most 
other WCPSS schools. It can be concluded that there are other WCPSS teachers and schools 
whose students are also producing less than their best work. 

Behaviors of Top Teachers 

Although both top and bottom teachers agreed that their students had weak prerequisite skills, 
the attitude toward these students and the approach for remediating was quite different. The top 
teachers and schools believed that they could remediate within a spiraled curriculum of new 
material and that their students could be successful learning new material while reviewing old 
material. One teacher at a top school said, “Our students are proud of how they do.” 

Effective teachers in this study also had a plan before the school year began. If the plan was a 
school wide one, then the entire school was more likely to be effective and new teachers were 
supported to success quickly. The plan was aligned to the NCSCS,with textbooks as one among 
many possible resources. Teachers planned the objective for the day by a concept or goal 
description and not by a chapter number or name from a textbook. The most effective Algebra I 
teachers spent most of their planning time together discussing pacing. The course plan was 
regularly updated to reflect the pacing needs of students and curriculum. 

When asked about important topics in Algebra I, highly effective teachers agreed that the 
teaching of problem solving, how to approach exercises in general, and concepts behind 
questions were of most importance. None of these is specifically named in the NCSCS, yet top 
teachers saw these as key to success in mastering the curriculum and in being prepared for future 
mathematics courses. Bottom teachers did not think that they had time to work on problem 
solving. They were too busy insisting that students memorize algorithms and work on skills. 

The thoughtful use of time and materials was observed in classrooms of top teachers. Among 
top teachers, there was an urgency to teach from bell to bell. Top teachers used opening 
questions and exercises to quickly review past skills and serve as advanced organizers for new 
material. Top teachers watched time carefully, using the bulk of the period on new material, yet 
leaving the final five to nine minutes for a wrap-up. Bottom teachers were observed using the 
majority of their time on old material and wrap-up. One bottom teacher was observed allowing 
students 20 minutes at the beginning to check homework answers with a key on the overhead 
projector, and then, after an interesting 50 minute lesson, the class spent the last 20 minutes 
copying the next homework assignment. Despite the interesting lesson there was much off-task 
behavior at the beginning and end of class. 
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It can be concluded that both top and bottom teachers reported more use of Marzano strategies 
than was observed. Most teachers were providing feedback, as well as asking questions, 
providing cues, and using advanced organizers. Questioning was at a much higher level in the 
classes of top teachers. They probed their students’ conceptual understanding and the whys 
behind answers; bottom teachers questioned the steps needed in skills. Top teachers also 
involved more students during their questioning. Top teachers provided advanced organizers in 
their warm-up exercises. 

The use of homework was observed in most classes, but only the top teachers were observed 
taking the time to give purpose to the homework. Top teachers explained why they chose a 
homework assignment and how it would reinforce or introduce a concept. They used homework 
questions within a lesson to review or explain concepts. Bottom teachers checked to see whether 
the assignment was done but did not use homework meaningfully during the day’s lesson. 

Only one teacher was observed providing the day’s objective, yet eight said that they did so 
daily. There seemed to be an implied objective that could be observed from the activities and 
exercises of the day, but was not clearly stated in words of the NCSCS. Since there was no 
clearly stated objective, the wrap-ups at the end of class lacked a feeling of closure that would 
help students make sense of the time that they had spent in class and on their homework. 

Only 3 of 15 teachers generated and tested hypotheses. Only 5 of 15 teachers used similarities 
and differences. Only 4 of 15 summarized, and only four of eight top and two of seven bottom 
teachers used nonlinguistic representations. Marzano (2001) found all of these instructional 
strategies to be highly effective in promoting student learning, and these techniques are promoted 
by the National Council of Teachers of Mathematics. All four of these strategies can be used 
daily in any mathematics classroom. There was room for improvement in all the teachers in the 
study. 

Summary 

The following conclusions were made of the most effective teachers: 

• They actively participated in developing and using an instructional plan aligned to the 
standard course of study. 

• They planned with other teachers. 

• They were concerned about pacing. 

• They used a spiraled curriculum that introduced new material while remediating 
prerequisites. 

• They created a structured but positive classroom culture. 

• They emphasized problem solving. 

• They had a positive attitude toward student performance. 

• They used materials and class time thoughtfully. 
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The following conclusions were made about the most effective schools: 

• They had a strong experienced course leader. 

• They had a school wide plan consistent across classes aligned to the standard course of study 
with six components: 

► materials ready for the entire year, 

► remediation within new material, 

► emphasis on problem solving and processes, 

► spiraled curriculum, 

► support structures for teachers with special consideration of new teachers, and 

► materials used thoughtfully. 

DISCUSSION AND IMPLICATIONS 

The evidence is clear from both this study and the previous biology study that there are teacher 
and school models of effectiveness that can serve as exemplars for other teachers and schools 
within our school system (Haynie, 2006). Most schools have pockets of excellence and other 
areas in need of improvement. This study identified some areas where all teachers are falling 
short of excellence. 

The overall state of Algebra I in WCPSS is healthy and performing well, yet there is much room 
for improvement. It is clear from this study that the attitude, focus, and commitment of the top 
teachers produced high performance in their students and that all students can be successful in 
Algebra I given the right environment. 

The bottom teachers felt that they could not go into new material without first reteaching the 
prerequisite skills. They viewed Algebra I as a fixed linear progression. This attitude often led 
to a negative classroom atmosphere in which students felt inadequate and defeated. The students 
were bored with the old material and had accepted that they could not do any better. Some 
bottom teachers also voiced negative attitudes toward their students. These bottom teachers 
characterized their students as noisy, immature, dependent, unable to think on their own, or 
having poor study skills. Top teachers created a positive classroom culture with a positive 
attitude toward student ability to perform well. Interestingly, these top teachers also used humor. 

Accepting, studying, and facilitating the NCSCS is crucial to student success. A bottom teacher 
was observed providing a rich, comfortable mathematics environment where students were 
actively engaged in their learning, but when introducing a new topic she spent two-thirds of a 
class period on Algebra II material that would not be tested in Algebra I. She said to the 

students, "When you see this on your EOC exam, I want you to ". She was unaware of the 

Algebra I curriculum requirements and used class time unwisely. Enriching the curriculum 
beyond the minimum requirements is an acceptable practice and encouraged with advanced 
students, but the basic curriculum must be taught first and teachers need to know when they are 
enriching it. Top teachers had a plan that aligned to the curriculum. 

Content knowledge is also very important. The top teachers were able to use high-level 
questioning because of their depth of understanding of algebra. They responded to student 
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questions with greater clarity than bottom teachers. Developing depth of content knowledge 
takes a commitment to continual study and reflection. 

Teaching is a complex profession. The top teachers of this study had more of the necessary 
components present in their classrooms. Most bottom teachers had parts, but not the whole. 

Some had great content knowledge and instructional presentation but mismanaged their time or 
did not know the curriculum well. Others managed time well, but were off curriculum or had 
weak content knowledge. Some bottom teachers did not believe that their students could learn. 

The following recommendations for improvement are made to all Algebra I teachers: 

• Study the standard course of study and EOC testing program. 

• Study goal summaries. 

• Study residual rosters. 

• Plan with other teachers. 

• Understand and implement Marzano strategies. 

• Use and/or develop a school plan for Algebra 1 . 

• Study mathematics to increase depth of content knowledge. 

• Reflect on performance data and instructional practice. 

This study found, as did the biology study, that there is a school effect in successful performance 
of students in their courses. The school effect is department-driven. Most schools have some 
departments working together and performing better than other departments in the school. 
Although a teacher can be successful in a weak department, some schools have plans that support 
the success of all teachers including those new to the profession. 

One of the greatest concerns voiced by many of the teachers of this study was the high turnover 
of teachers in Algebra I and the high number of new teachers assigned to teach Algebra I. Only 
26% of the 157 Algebra I teachers in 2005-06 had taught Algebra I consecutively the three 
previous years. Experienced teachers often request to teach higher-level courses, while new 
teachers request Algebra I, thinking that it is the most familiar material. Higher-level courses are 
used as rewards for longevity. Since Algebra I is a required course for all students and the 
Algebra I EOC must be passed, it is important to have a school wide assignment plan that 
ensures students have good teachers. A new teacher can be good if supported by experienced 
teachers and if there is a curriculum-aligned school pacing guide. 

It was also observed that the bottom teachers had students with an overall lower 8th-grade 
mathematics scale score average, and an overall higher standard deviation of these eight grade 
scores. This would indicate that these teachers have, on the whole, a more challenging student 
assignment than the top teachers. Since observations showed the top teachers to be using better 
teaching practices and to be better overall teachers, it could be concluded that often top teachers 
were being rewarded with better teaching assignments than the bottom teachers. 

Top schools have found a way to schedule mathematics teachers that ensures stability of 
leadership and experience in Algebra I, while also giving teachers an opportunity to grow in 
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higher-level eourses. Sehools must eollaborate with teaehers to eonvinee them of the importanee 
of Algebra I and together develop an assignment plan of sueeess. 

The following reeommendations are made to school-based leadership; 

• Develop a school plan that aligns to the standard course of study, emphasizes problem 
solving, and supports new teachers. 

• Support meaningful common planning for teachers. 

• Share results data with teachers including effectiveness rosters and indices. 

• Develop a scheduling plan that maintains stability in Algebra I while adjusting to 
performance results over time and giving teachers opportunities to grow. 

• Support a school culture that promotes open discourse at all levels. 

This study (and the earlier biology study) also identified areas where central district leadership 
can help facilitate school improvement. Since central leadership has access to all schools and 
time dedicated to supporting schools, their main function can be sharing and explaining the best 
practices observed. 

The following recommendations are made to district leadership; 

• Make observations of most effective schools and teachers that can be shared district wide. 

• Provide workshops on implementing Marzano strategies. 

• Support school wide improvement efforts based on top school models. 

• Support teacher improvement efforts. 

• Provide data to teachers and schools on their effectiveness. 

This Algebra I study added valuable data to that collected in the biology study. Teacher 
effectiveness studies will continue in WCPSS and overall conclusions will crystallize in time. 
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